{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "7a8e33c4",
   "metadata": {},
   "source": [
    "Copyright (c) MONAI Consortium  \n",
    "Licensed under the Apache License, Version 2.0 (the \"License\");  \n",
    "you may not use this file except in compliance with the License.  \n",
    "You may obtain a copy of the License at  \n",
    "&nbsp;&nbsp;&nbsp;&nbsp;http://www.apache.org/licenses/LICENSE-2.0  \n",
    "Unless required by applicable law or agreed to in writing, software  \n",
    "distributed under the License is distributed on an \"AS IS\" BASIS,  \n",
    "WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.  \n",
    "See the License for the specific language governing permissions and  \n",
    "limitations under the License."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "8a0aaced",
   "metadata": {},
   "source": [
    "# Lazy resampling benchmark\n",
    "\n",
    "In this notebook, we used 3D spleen segmentation task to show our lazy-resampling benchmark. Our results include the following two main parts.\n",
    "- Time spent on each transform and the total amount of data preparation in the lazy and non-lazy mode.\n",
    "- End-to-end time comparison"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "626a7bfc",
   "metadata": {},
   "source": [
    "## Setup environment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "9beac76a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python -c \"import monai\" || pip install -q \"monai-weekly[nibabel, tqdm, gdown]\""
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "1b74510e",
   "metadata": {},
   "source": [
    "## Setup imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cdc814c0",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import time\n",
    "import torch\n",
    "import tempfile\n",
    "import monai\n",
    "import monai.transforms as mt\n",
    "from monai.utils import WorkflowProfiler\n",
    "from monai.apps import download_and_extract\n",
    "\n",
    "from monai.config import print_config\n",
    "\n",
    "print_config()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "80fd97bc",
   "metadata": {},
   "source": [
    "# Setup data directory\n",
    "You can specify a directory with the `MONAI_DATA_DIRECTORY` environment variable.  \n",
    "This allows you to save results and reuse downloads.  \n",
    "If not specified a temporary directory will be used."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "6cd43b69",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/workspace/Data\n"
     ]
    }
   ],
   "source": [
    "directory = os.environ.get(\"MONAI_DATA_DIRECTORY\")\n",
    "if directory is not None:\n",
    "    os.makedirs(directory, exist_ok=True)\n",
    "root_dir = tempfile.mkdtemp() if directory is None else directory\n",
    "print(root_dir)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "806db6d7",
   "metadata": {},
   "source": [
    "## Download dataset and prepare the utilities\n",
    "\n",
    "This section downloads and extracts the dataset.\n",
    "\n",
    "The dataset comes from http://medicaldecathlon.com/.\n",
    "\n",
    "`TraceObjectsOnly` is used to narrow the scope of the trace to top-level transforms only."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "4255c4b0",
   "metadata": {},
   "outputs": [],
   "source": [
    "class TraceObjectsOnly:\n",
    "    def __init__(self, objects):\n",
    "        self.objects = objects\n",
    "\n",
    "    def __call__(self, frame):\n",
    "        self_obj = frame.f_locals.get(\"self\", None)\n",
    "        return frame.f_code.co_name == \"__call__\" and self_obj in self.objects\n",
    "\n",
    "\n",
    "resource = \"https://msd-for-monai.s3-us-west-2.amazonaws.com/Task09_Spleen.tar\"\n",
    "md5 = \"410d4a301da4e5b2f6f86ec3ddba524e\"\n",
    "\n",
    "compressed_file = os.path.join(root_dir, \"Task09_Spleen.tar\")\n",
    "data_dir = os.path.join(root_dir, \"Task09_Spleen\")\n",
    "if not os.path.exists(data_dir):\n",
    "    download_and_extract(resource, compressed_file, root_dir, md5)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "49e8149b",
   "metadata": {},
   "source": [
    "## Transform Profiling Comparison"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "fe5f7a0f",
   "metadata": {},
   "source": [
    "`transform_list` shows the transform chain.\n",
    "- `LoadImaged` loads the brats MRI images and labels from NIfTI format files.\n",
    "- `EnsureChannelFirstd` ensures the original data to construct \"channel first\" shape.\n",
    "- `ConvertToMultiChannelBasedOnBratsClassesd` convert labels to multi channels based on brats classes.\n",
    "- `Orientationd` unifies the data orientation based on the affine matrix.\n",
    "- `Spacingd` adjusts the spacing by `pixdim=(1., 1., 1.)` based on the affine matrix.\n",
    "- `RandSpatialCropd` crop the image and label to [224, 224, 144] at a random position as center.\n",
    "- `RandFlipd` randomly reverse the order of elements along the given spatial axis.\n",
    "- `NormalizeIntensityd` normalize the input based on calculated mean and std.\n",
    "- `RandScaleIntensityd` randomly scale the intensity of input image.\n",
    "- `RandShiftIntensityd` randomly shift intensity with randomly picked offset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "e3fd7197",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "monai.transforms.io.dictionary LoadImaged.__init__:image_only: Current default value of argument `image_only=False` has been deprecated since version 1.1. It will be changed to `image_only=True` in version 1.3.\n"
     ]
    }
   ],
   "source": [
    "transform_list = [\n",
    "    # load 4 Nifti images and stack them together\n",
    "    mt.LoadImaged(keys=[\"image\", \"label\"]),\n",
    "    mt.EnsureChannelFirstd(keys=\"image\"),\n",
    "    mt.EnsureTyped(keys=[\"image\", \"label\"]),\n",
    "    mt.ConvertToMultiChannelBasedOnBratsClassesd(keys=\"label\"),\n",
    "    mt.Orientationd(keys=[\"image\", \"label\"], axcodes=\"RAS\"),\n",
    "    mt.Spacingd(\n",
    "        keys=[\"image\", \"label\"],\n",
    "        pixdim=(1.0, 1.0, 1.0),\n",
    "        mode=(\"bilinear\", \"nearest\"),\n",
    "    ),\n",
    "    mt.RandSpatialCropd(keys=[\"image\", \"label\"], roi_size=[224, 224, 144], random_size=False),\n",
    "    mt.RandFlipd(keys=[\"image\", \"label\"], prob=1.0, spatial_axis=0),\n",
    "    mt.RandFlipd(keys=[\"image\", \"label\"], prob=1.0, spatial_axis=1),\n",
    "    mt.RandFlipd(keys=[\"image\", \"label\"], prob=1.0, spatial_axis=2),\n",
    "    mt.NormalizeIntensityd(keys=\"image\", nonzero=True, channel_wise=True),\n",
    "    mt.RandScaleIntensityd(keys=\"image\", factors=0.1, prob=1.0),\n",
    "    mt.RandShiftIntensityd(keys=\"image\", offsets=0.1, prob=1.0),\n",
    "]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "e1b8d61a",
   "metadata": {},
   "source": [
    "### The preprocessing pipeline evaluated non-lazily"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "72e3c8f3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "non-lazy preprocessing time: 327.38880467414856\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Count</th>\n",
       "      <th>Total Time (s)</th>\n",
       "      <th>Avg</th>\n",
       "      <th>Std</th>\n",
       "      <th>Min</th>\n",
       "      <th>Max</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>LoadImaged.__call__</th>\n",
       "      <td>388</td>\n",
       "      <td>214.600060</td>\n",
       "      <td>0.553093</td>\n",
       "      <td>0.014112</td>\n",
       "      <td>0.519402</td>\n",
       "      <td>0.600372</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>NormalizeIntensityd.__call__</th>\n",
       "      <td>388</td>\n",
       "      <td>51.334329</td>\n",
       "      <td>0.132305</td>\n",
       "      <td>0.026704</td>\n",
       "      <td>0.096193</td>\n",
       "      <td>0.192629</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RandFlipd.__call__</th>\n",
       "      <td>1164</td>\n",
       "      <td>21.008026</td>\n",
       "      <td>0.018048</td>\n",
       "      <td>0.002203</td>\n",
       "      <td>0.016785</td>\n",
       "      <td>0.069709</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ConvertToMultiChannelBasedOnBratsClassesd.__call__</th>\n",
       "      <td>388</td>\n",
       "      <td>5.235506</td>\n",
       "      <td>0.013494</td>\n",
       "      <td>0.001316</td>\n",
       "      <td>0.012556</td>\n",
       "      <td>0.027214</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Spacingd.__call__</th>\n",
       "      <td>388</td>\n",
       "      <td>4.345852</td>\n",
       "      <td>0.011201</td>\n",
       "      <td>0.006300</td>\n",
       "      <td>0.009089</td>\n",
       "      <td>0.115761</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RandShiftIntensityd.__call__</th>\n",
       "      <td>388</td>\n",
       "      <td>4.018844</td>\n",
       "      <td>0.010358</td>\n",
       "      <td>0.006234</td>\n",
       "      <td>0.009085</td>\n",
       "      <td>0.057106</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RandScaleIntensityd.__call__</th>\n",
       "      <td>388</td>\n",
       "      <td>3.786071</td>\n",
       "      <td>0.009758</td>\n",
       "      <td>0.003357</td>\n",
       "      <td>0.008987</td>\n",
       "      <td>0.049616</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Orientationd.__call__</th>\n",
       "      <td>388</td>\n",
       "      <td>1.406409</td>\n",
       "      <td>0.003625</td>\n",
       "      <td>0.010838</td>\n",
       "      <td>0.001987</td>\n",
       "      <td>0.120482</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RandSpatialCropd.__call__</th>\n",
       "      <td>388</td>\n",
       "      <td>0.611028</td>\n",
       "      <td>0.001575</td>\n",
       "      <td>0.001485</td>\n",
       "      <td>0.001260</td>\n",
       "      <td>0.030637</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>EnsureChannelFirstd.__call__</th>\n",
       "      <td>388</td>\n",
       "      <td>0.083966</td>\n",
       "      <td>0.000216</td>\n",
       "      <td>0.000031</td>\n",
       "      <td>0.000188</td>\n",
       "      <td>0.000482</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>EnsureTyped.__call__</th>\n",
       "      <td>388</td>\n",
       "      <td>0.081848</td>\n",
       "      <td>0.000211</td>\n",
       "      <td>0.001671</td>\n",
       "      <td>0.000117</td>\n",
       "      <td>0.033091</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                    Count  Total Time (s)  \\\n",
       "LoadImaged.__call__                                   388      214.600060   \n",
       "NormalizeIntensityd.__call__                          388       51.334329   \n",
       "RandFlipd.__call__                                   1164       21.008026   \n",
       "ConvertToMultiChannelBasedOnBratsClassesd.__call__    388        5.235506   \n",
       "Spacingd.__call__                                     388        4.345852   \n",
       "RandShiftIntensityd.__call__                          388        4.018844   \n",
       "RandScaleIntensityd.__call__                          388        3.786071   \n",
       "Orientationd.__call__                                 388        1.406409   \n",
       "RandSpatialCropd.__call__                             388        0.611028   \n",
       "EnsureChannelFirstd.__call__                          388        0.083966   \n",
       "EnsureTyped.__call__                                  388        0.081848   \n",
       "\n",
       "                                                         Avg       Std  \\\n",
       "LoadImaged.__call__                                 0.553093  0.014112   \n",
       "NormalizeIntensityd.__call__                        0.132305  0.026704   \n",
       "RandFlipd.__call__                                  0.018048  0.002203   \n",
       "ConvertToMultiChannelBasedOnBratsClassesd.__call__  0.013494  0.001316   \n",
       "Spacingd.__call__                                   0.011201  0.006300   \n",
       "RandShiftIntensityd.__call__                        0.010358  0.006234   \n",
       "RandScaleIntensityd.__call__                        0.009758  0.003357   \n",
       "Orientationd.__call__                               0.003625  0.010838   \n",
       "RandSpatialCropd.__call__                           0.001575  0.001485   \n",
       "EnsureChannelFirstd.__call__                        0.000216  0.000031   \n",
       "EnsureTyped.__call__                                0.000211  0.001671   \n",
       "\n",
       "                                                         Min       Max  \n",
       "LoadImaged.__call__                                 0.519402  0.600372  \n",
       "NormalizeIntensityd.__call__                        0.096193  0.192629  \n",
       "RandFlipd.__call__                                  0.016785  0.069709  \n",
       "ConvertToMultiChannelBasedOnBratsClassesd.__call__  0.012556  0.027214  \n",
       "Spacingd.__call__                                   0.009089  0.115761  \n",
       "RandShiftIntensityd.__call__                        0.009085  0.057106  \n",
       "RandScaleIntensityd.__call__                        0.008987  0.049616  \n",
       "Orientationd.__call__                               0.001987  0.120482  \n",
       "RandSpatialCropd.__call__                           0.001260  0.030637  \n",
       "EnsureChannelFirstd.__call__                        0.000188  0.000482  \n",
       "EnsureTyped.__call__                                0.000117  0.033091  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "monai.utils.set_determinism(24)\n",
    "\n",
    "train_transform = mt.Compose(transform_list)\n",
    "train_ds = monai.apps.DecathlonDataset(\n",
    "    root_dir=root_dir,\n",
    "    task=\"Task01_BrainTumour\",\n",
    "    transform=train_transform,\n",
    "    section=\"training\",\n",
    "    download=True,\n",
    "    cache_rate=0.0,\n",
    ")\n",
    "data_loader = monai.data.DataLoader(train_ds, batch_size=1, shuffle=True)\n",
    "\n",
    "with WorkflowProfiler(TraceObjectsOnly(transform_list)) as wp:\n",
    "    time_start = time.time()\n",
    "    for _item_lazy in data_loader:\n",
    "        pass\n",
    "    time_process = time.time() - time_start\n",
    "\n",
    "print(f\"non-lazy preprocessing time: {time_process}\")\n",
    "wp.get_times_summary_pd()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "7de2e892",
   "metadata": {},
   "source": [
    "### The preprocessing pipeline evaluated lazily"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "116d6e84",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "lazy preprocessing time: 301.8494665622711\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Count</th>\n",
       "      <th>Total Time (s)</th>\n",
       "      <th>Avg</th>\n",
       "      <th>Std</th>\n",
       "      <th>Min</th>\n",
       "      <th>Max</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>LoadImaged.__call__</th>\n",
       "      <td>388</td>\n",
       "      <td>210.230422</td>\n",
       "      <td>0.541831</td>\n",
       "      <td>0.011048</td>\n",
       "      <td>0.509415</td>\n",
       "      <td>0.589100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>NormalizeIntensityd.__call__</th>\n",
       "      <td>388</td>\n",
       "      <td>51.072924</td>\n",
       "      <td>0.131631</td>\n",
       "      <td>0.027062</td>\n",
       "      <td>0.099262</td>\n",
       "      <td>0.195547</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RandShiftIntensityd.__call__</th>\n",
       "      <td>388</td>\n",
       "      <td>5.196085</td>\n",
       "      <td>0.013392</td>\n",
       "      <td>0.008437</td>\n",
       "      <td>0.008982</td>\n",
       "      <td>0.037699</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ConvertToMultiChannelBasedOnBratsClassesd.__call__</th>\n",
       "      <td>388</td>\n",
       "      <td>5.149661</td>\n",
       "      <td>0.013272</td>\n",
       "      <td>0.001081</td>\n",
       "      <td>0.012500</td>\n",
       "      <td>0.023830</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RandScaleIntensityd.__call__</th>\n",
       "      <td>388</td>\n",
       "      <td>3.762584</td>\n",
       "      <td>0.009697</td>\n",
       "      <td>0.000706</td>\n",
       "      <td>0.009134</td>\n",
       "      <td>0.022499</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Orientationd.__call__</th>\n",
       "      <td>388</td>\n",
       "      <td>1.879595</td>\n",
       "      <td>0.004844</td>\n",
       "      <td>0.006223</td>\n",
       "      <td>0.001772</td>\n",
       "      <td>0.023112</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Spacingd.__call__</th>\n",
       "      <td>388</td>\n",
       "      <td>1.359067</td>\n",
       "      <td>0.003503</td>\n",
       "      <td>0.001040</td>\n",
       "      <td>0.003040</td>\n",
       "      <td>0.020699</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RandFlipd.__call__</th>\n",
       "      <td>1164</td>\n",
       "      <td>0.802183</td>\n",
       "      <td>0.000689</td>\n",
       "      <td>0.000109</td>\n",
       "      <td>0.000561</td>\n",
       "      <td>0.002725</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RandSpatialCropd.__call__</th>\n",
       "      <td>388</td>\n",
       "      <td>0.275304</td>\n",
       "      <td>0.000710</td>\n",
       "      <td>0.000167</td>\n",
       "      <td>0.000570</td>\n",
       "      <td>0.003660</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>EnsureChannelFirstd.__call__</th>\n",
       "      <td>388</td>\n",
       "      <td>0.084098</td>\n",
       "      <td>0.000217</td>\n",
       "      <td>0.000024</td>\n",
       "      <td>0.000187</td>\n",
       "      <td>0.000323</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>EnsureTyped.__call__</th>\n",
       "      <td>388</td>\n",
       "      <td>0.049539</td>\n",
       "      <td>0.000128</td>\n",
       "      <td>0.000009</td>\n",
       "      <td>0.000118</td>\n",
       "      <td>0.000242</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                    Count  Total Time (s)  \\\n",
       "LoadImaged.__call__                                   388      210.230422   \n",
       "NormalizeIntensityd.__call__                          388       51.072924   \n",
       "RandShiftIntensityd.__call__                          388        5.196085   \n",
       "ConvertToMultiChannelBasedOnBratsClassesd.__call__    388        5.149661   \n",
       "RandScaleIntensityd.__call__                          388        3.762584   \n",
       "Orientationd.__call__                                 388        1.879595   \n",
       "Spacingd.__call__                                     388        1.359067   \n",
       "RandFlipd.__call__                                   1164        0.802183   \n",
       "RandSpatialCropd.__call__                             388        0.275304   \n",
       "EnsureChannelFirstd.__call__                          388        0.084098   \n",
       "EnsureTyped.__call__                                  388        0.049539   \n",
       "\n",
       "                                                         Avg       Std  \\\n",
       "LoadImaged.__call__                                 0.541831  0.011048   \n",
       "NormalizeIntensityd.__call__                        0.131631  0.027062   \n",
       "RandShiftIntensityd.__call__                        0.013392  0.008437   \n",
       "ConvertToMultiChannelBasedOnBratsClassesd.__call__  0.013272  0.001081   \n",
       "RandScaleIntensityd.__call__                        0.009697  0.000706   \n",
       "Orientationd.__call__                               0.004844  0.006223   \n",
       "Spacingd.__call__                                   0.003503  0.001040   \n",
       "RandFlipd.__call__                                  0.000689  0.000109   \n",
       "RandSpatialCropd.__call__                           0.000710  0.000167   \n",
       "EnsureChannelFirstd.__call__                        0.000217  0.000024   \n",
       "EnsureTyped.__call__                                0.000128  0.000009   \n",
       "\n",
       "                                                         Min       Max  \n",
       "LoadImaged.__call__                                 0.509415  0.589100  \n",
       "NormalizeIntensityd.__call__                        0.099262  0.195547  \n",
       "RandShiftIntensityd.__call__                        0.008982  0.037699  \n",
       "ConvertToMultiChannelBasedOnBratsClassesd.__call__  0.012500  0.023830  \n",
       "RandScaleIntensityd.__call__                        0.009134  0.022499  \n",
       "Orientationd.__call__                               0.001772  0.023112  \n",
       "Spacingd.__call__                                   0.003040  0.020699  \n",
       "RandFlipd.__call__                                  0.000561  0.002725  \n",
       "RandSpatialCropd.__call__                           0.000570  0.003660  \n",
       "EnsureChannelFirstd.__call__                        0.000187  0.000323  \n",
       "EnsureTyped.__call__                                0.000118  0.000242  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "monai.utils.set_determinism(24)\n",
    "\n",
    "overrides = {\n",
    "    \"image\": {\"mode\": \"bilinear\", \"padding_mode\": \"border\", \"dtype\": torch.float32},\n",
    "    \"label\": {\"mode\": 0, \"padding_mode\": \"nearest\", \"dtype\": torch.uint8},\n",
    "}\n",
    "train_transform = mt.Compose(transform_list, lazy=True, overrides=overrides)\n",
    "train_ds = monai.apps.DecathlonDataset(\n",
    "    root_dir=root_dir,\n",
    "    task=\"Task01_BrainTumour\",\n",
    "    transform=train_transform,\n",
    "    section=\"training\",\n",
    "    download=True,\n",
    "    cache_rate=0.0,\n",
    ")\n",
    "data_loader = monai.data.DataLoader(train_ds, batch_size=1, shuffle=True)\n",
    "with WorkflowProfiler(TraceObjectsOnly(transform_list)) as wp:\n",
    "    time_start = time.time()\n",
    "    for _item_lazy in data_loader:\n",
    "        pass\n",
    "    time_process = time.time() - time_start\n",
    "\n",
    "print(f\"lazy preprocessing time: {time_process}\")\n",
    "wp.get_times_summary_pd()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "48ba0615",
   "metadata": {},
   "source": [
    "## End-to-end workflow Profiling Comparison\n",
    "\n",
    "https://github.com/Project-MONAI/tutorials/blob/main/3d_segmentation/spleen_segmentation_3d.ipynb contains the complete workflow. Here we should modify it in two places:\n",
    "- `transform_list` should be used in place of the 'train_transforms' and 'val_transforms'.\n",
    "- Use regular Dataset instead of CacheDataset for training and validation process."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "3173dbeb",
   "metadata": {},
   "source": [
    "### Total time and every epoch time comparison\n",
    "![lazy_benchmark_total_epoch_time_comparison](../figures/lazy_benchmark_total_epoch_time_comparison.png)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "dbe52762",
   "metadata": {},
   "source": [
    "### Performance comparison\n",
    "The end-to-end pipeline was benchmarked on a A100 80G GPU.\n",
    "\n",
    "Lazy mode training: best_metric: Dice 0.7970 at epoch: 65 total time: 41296.6880s\n",
    "\n",
    "Non-lazy mode training: best_metric: Dice 0.7955 at epoch: 68 total time: 42887.6809s"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
